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Abstract 

iTiis paper examines the problems of centralized 
academic indicator systems in light of thrusts toward 
site management of schools. Problems with current 
practice are examined in the framework of critical 
inquiry. Alternatives to current accountability 
guidelines are presented which harmonize positivism with 
critical inquiry, while respecting both local and state 
needs for accountability information. 
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Introduction 

This paper examines the conflict between 
centralized school assessment and thrusts toward site- 
management and draws implications for local research 
agendas. Decentralized decision-making in public 
schools has taken the country by storm. Texas is no 
exception to this trend where, in 1991, site-based 
management was signed into law. in contrast, the manner 
in which schools are held accountable for their 
performance remains highly centralized. The Texas 
Academic Excellence Indicator System (AElS) is a top- 
down mandated set of school performance elements. As 
such, the success of any localized effort to improve 
education is judged by 67 state -mandated indicators, 
only 27 of which are directly related to student 
performance, in spite of concerted efforts to assess 
performance in the 1,054 school districts in Texas, the 
results of the AElS becomes "filtered unpredictably 
through multiple intermediate layers of discretion 
shaped by preexisting cultures and agendas." (Clune, 
1993, p. 234) 

I use a focused synthesis methodologi/ (Majcharzak, 
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1984) of policy research to examine the paradox of 
"statist -centralized" data collection (Clune, 1993) in 
the presence of decentralized decision making. First I 
describe the AEIS as a centralized assessment device and 
how it fits into an economic model for school 
assessment. Next I discuss the legislative incarnation 
of Texas’ site-based management, followed by a 
discussion on how it can ameliorate problems associated 
with employing a strictly economic model of school 
assessment. Finally I make recommendations for research 
practices at the local level which respect the value- 
laden nature of local needs while still embracing a 
positivistic approach that appeases major stakeholders. 

The AEIS as Production- Function 

Many states have adopted high- stakes student 
testing, extending the application of the production- 
function paradigm in school accountability. High stakes 
tests play to the outcomes side of this model, as 
minimum standards of student performance result in 
rewards or consequences. For state policy makers this 
approach permits the use of an economic model to assess 
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schools. Ferguson's treatise for "how and why money 
matters" (1991, p 465) in education in the Harvard 
Journal on Legislation is an example of this kind of 
approach. Using data aggregated at school, district, 
and state levels, outcomes such as test scores (Dolan & 
Schmidt, 1987; Sebold & Dato, i98l) are measured against 
various inputs, such as expenditures and student 
characteristics . 

The AEIS is such an accountability measure, 
describing student ability on fourteen dependent 
variables. When linked with independent variables 
related to student demographics, the AEIS report 
spans sixty- seven columns across a spreadsheet v/indow. 
The major student -related dependent variables, as 
recommended to the State Board of Education for 1992-93 
and beyond, are: 

• Texas Assessment of Academic Skills (TAAS) % Passing 

• Texas Assessment of Academic Skills (TAAS) % Mastery 

• Physical Fitness and Health 

• Computer Literacy 

• Second Language Proficiency 

• Norm- ref erenced Assessment Program 
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• Texas Academic Skills Program (predicted success) 

• Graduation Rate 

• Advanced Course Conpletion 

• Student Attendance 

• Promotion /Re tent ion 

• sat/act Scores 

• Completion of Recommended Preparatory Program 

• Dropout Rate 

The student -related independent variables are: 
ethnicity, whether or not the student is " limit ed- 
English proficient, if a student is considered from a 
"low- income" family, and whether the student is expected 
to attend college or not. 

To coirpare campuses to themselves over time, and 
different campuses to each other, the state uses a 
strictly positivistic approach, engaging in v/hat any 
educational research text will tell you is correlational 
research (eg: McMillan & Schumacher, 1986) . 

In an extreme example that approaches social 
Darwinsim, the state has held all school buildings 
accountable to high stakes sanctions based on only one 
variable (Scharrer, 1993), percentage of students 
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passing all three portions of the TAAS test. -Holding a 
vast number of schools, ostensibly charged with their 
own destiny, to such a restricted list of factors 
dismisses local level capabilities to conduct research 
on "what works . " 

While the state has a legitimate need to collect 
accountability data, current evaluation practices stifle 
the development of innovative methods of self-study by 
school personnel. True, the current set of AEIS 
indicators can do a good job of providing a "snapshot" 
or "yearbook picture" of how schools are doing, but they 
do little in the v?ay of describing how classroom 
practice needs to change to improve the schools that are 
having a "bad- hair day" when their picture is taken. 
Knowledge that is verified by snapshots of complex 
issues have a focus that is much too simple and narrow. 
Annual reports that emanate from systems such as the 
AEIS are devoid of the context needed to comprehend the 
complexities of the classroom. Further, it would be 
remiss to prescribe from this data generalizable 
procedures for effective teaching (Kincheloe, 1991) . 
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Problems with Production- Function Approaches 

Monk (1993) notes that this research remains as 
"fundamentally primitive black-box formulations where 
analysts have made little progress toward modeling what 
makes education distinct from other types of productions 
more typically studied using production function 
techniques (Monk, 1993 p. 309) One problem is that 
traditional indicators (gender, socio-economic status, 
etc.) do not lend themselves to this type of assessment. 
Further, Rivlin (1973) finds it is difficult to 
extricate the variance of student input in measured 
outcomes. Where, then, should outcomes analyses take 
place? Senge notes that "learning to see underlying 
structures rather than events is a starting point..." 
(1990, p. 65) . By using a lens of critical inquiry, 
greater latitude could to be granted to local 
practitioners to set a research agenda respecting the 
value- laden nature of local needs while still embracing 
a positivistic approach that appeases major 
stakeholders. This is where the infrastructure of site- 
based management provides a platform for local research 
agendas to validate local indicators of student success. 
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Site Based Management: what's Critical Inquiry 

got to do with it? 

Site based management is a legislated practice in 
Texas, In May of 1991 House Bill 2885 mandated that 
each school shall develop and irt^jlement a plan for site- 
based decision making. However, in spite of this 
legislated freedom, many teachers and administrators 
feel free to vary curriculxom and methods only to the 
extent that it improves centralized accountability 
measures. As such, data on "what works" tends only to 
come from annual AEIS reports. Greater mechanisms need 
to be in place that provide a climate of self scrutiny 
on the part of the local schools, while it is 
paradoxical that the state dictates local decision- 
making procedures, they are the first to admit to it. 

For the Texas Education Agency, site-based decision 
making "...is the message that it is the business of 
local communities and their respective school systems to 
determine how educational programs will be structured -- 
and that the state's role is to establish the standards 
for performance, determine district accountability with 
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respect to state standards, and coordinate technical 
support to assist local districts in their efforts to 
improve students performance. (TEA, 1992, p 1.3)" 

This paradox remains a difficult alliance. It is 
the business of determining how local educational 
programs will be structured - through local research 
efforts - that I wish to address. As noted above, the 
AEIS does little to provide local educational agencies 
with information on how to improve instructional 
practice. I believe that the infrastructure of site- 
based management in every Texas school provides an 
excellent platform upon which tenets of critical inquiry 
may be used to begin a process of local institutional 
research on the part of teachers and principals. Site- 
based management forces local practitioners to gather 
regularly to discuss all aspects of their educational 
program. At these meetings agendas can be set to 
examine locally developed indicators that may explain 
student success. 

Critical Inquiry and Practitioners as Researchers 

Kincheloe (1991) and Lytle & Cochran- Smith (1992) 
use tenets of critical inquiry to support the notion 
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that local school practitioners can and should conduct 
and disseminate research. By conducting research on the 
local level, practitioners can fill the gaps left by 
centralized snapshots of their schools. Teachers and 
principals are able to examine locally relevant 
variables that may explain student success with greater 
clarity than traditional independent variables such as 
ethnicity and socioeconomic status. Questions about 
student dignity and happiness may be introduced, as 
could forays into the control of the conceptualization 
of teachers’ work (Kincheloe, 1991). Thus practitioner 
research is a way of generating both local knowledge and 
public knowledge about teaching and learning. According 
to Lytle & Cochran- Smith, teacher research is a 
systematic, intentional inquiry by teachers about their 
own school and classroom work. The term systen^atic 
refers to ordered ways of "recollecting, rethinking, and 
analyzing classroom events" (p. 450) ; "intentional" 
means that research is a planned activity. 

The nature of practitioner research can take on 
many forms. Lytle & Cochran -Smith proposed four 
categories of school practitioner research, ranging from 
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teachers' journals, brief and book length essays on 
classroom and school analysis, oral inquiries and 
discussions, and finally large small and larger scale 
studies not unlike university based classroom research. 
Locally Developed Measures 

I am arguing here, drawing upon dune’s (1992) 
notion for a split level indicator system, and 
Kincheloe’s (1991) and Lytle & Cochran- Smith ’ s (1992) 
notions of teacher research capabilities that there is 
both value in, and need for, the development of what I 
call "locally developed measures." Locally developed 
measures are academic indicators that are determined and 
validated by teachers and principals at the building 
level (or at the district level if appropriate, i.e., 
small districts) and are recognized by the state as part 
of an academic excellence indicator system. I believe 
it is in the best interest of the state education agency 
to support, encourage, and call for local education 
agencies to propose and validate locally developed 
measures of student success. The research methodologies 
utilized can vary according to the researcher. 

Approaches advocated by Kincheloe (1991) and Lytle & 
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Cochran- Smith (1992) are qualitative in nature. While 
there is great value in qualitative methodologies, 
methodological monism is not the answer to any question. 
I support Francis Schrag in saying, "let a hundred 
methodological flowers bloom" (1992 p 16) . Policies 
that allow local schools and districts to report site- 
based data based on locally developed and approved 
research agendas will enhance education by: 

• acknowledging the importance of local relevance to 
school improvement assessment; 

• acknowledging the professionalism of local 
practitioners; 

• enhancing the database of assessment and improvement 
indicators; and 

• provide rich opportunities for colleges of education 
to partner with schools to conduct research through 
research courses. 

Constraints and Enablers to Sanctioning Local 
Agendas 

The workload placed upon district research offices 
is already burgeoning in the wake of modifications and 
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additions to the AEIS and state testing schedules. 
Answering the call to the state's centralized data needs 
is a formidable task. This is just one constraint to 
successfully sanctioning local research agendas in a 
thousand school districts in Texas. Another constraint 
is the presence of the cult of the expert. It is 
difficult to convince those blessed with society's 
credentials that unadorned educators should be granted 
the power to generate new knowledge. I am reminded of a 
conversation I had with a colleague on the subject of 
empowering teachers to conduct their own research: 

"Yes, but do you really think they can do it?" my 
colleague remarked. 

There are, however, enablers to the sanctioning of 
locally developed measures. Tlie organizational 
structure to deliver practitioner training currently 
exists. Colleges and universities are in a perfect 
position to re -tool pre- service programs that stress 
professional development and the use of technology for 
the purpose of site- based research, in Texas, some 
programs are already in place, such as the Center for 
Pedagogy at the University of Texas at El Paso, one of 
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several centers for professional development funded by 
the Texas Education Agency. These centers, which 
already stress explicit -linkages with public schools 
through off canpus teaching and common training in 
computer technology, could easily incorporate relevant 
research sequences that stress methods appropriate for 
the creation and validation of locally developed 
measures . 

Summary and Conclusions 

The AEIS can only provide snapshot data at best for 
the schools of Texas. The information gleaned from the 
AEIS data provide general direction for educational 
improvement, but does little to guide specific 
interventions at the classroom level. Site-based 
management creates a forum for local practitioners to 
gather and discuss, through critical inquiry, the 
creation and validation of locally developed measures of 
student success. There are those that believe it is 
misguided to train teachers to conduct research that 
helps improve student outcomes on centralized goals 
(Elliot, 1989) . They believe that this contradicts the 
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true purpose of critical inquiry. Nonetheless, the road 
to true local empowerment must begin somewhere. As long 
as the state continues to fund education, it will be 
difficult to wrestle all educational control to the 
local level. Any amount of critical reflection and 
participation by local actors regarding educational 
inprovement is a step in the right direction. 
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